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(57) Abstract: An image processing system, for correlating shapes in multi-dimensional images (m-D), comprising image data 
processing means for estimating a similarity measure including computing means for: estimating two image signals (f(x), g(y)) 
representing shapes defined in respective windows (Wl, W2) in two multi-dimensional images; using a Hermite Transform (HT) 
applied to both said image signals for performing an evaluation of two first sets of scalar valued Hermite coefficients (fi, gi, FT, 
GI), from which a combination yields a transformed set of scalar valued Hermite coefficients {KI}; applying the inverse Hermite 
Transform (HT-1) to the transformed set of scalar valued Hermite coefficients {KI} to achieve the computation of a windowed 
correlation function (K(v)); and estimating the maximum of said windowed correlation function as the wanted similarity measure to 
correlate the shapes; and means for displaying the correlated shapes and/or processed images. This system has means for applying 
rotation, translation and scale change on the scalar valued Hermite coefficients of the image signals and the correlation function, 
instead of on the images. This system can correlate very complex shapes having parts that locally show different deformations from 
the global complex shapes. The invention also relates to an examination apparatus coupled to this image processing system. 
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"IMAGE PROCESSING SYSTEM AND MEDICAL EXAMINATION APPARATUS FOR 

CORRELATING FEATURES IN MEDICAL IMAGES" 

Description 

Field of the Invention 

5 The invention relates to an image processing system having data processing means for 

correlating features in medical images and to a medical examination apparatus having such 
an image processing system. The present invention relates to an image processing method 
having data processing steps, to be used in the system and apparatus. The invention also 
relates to a computer program product having instructions for carrying out the method steps. 

10 The invention finds its application in the field of medical imaging and, more especially, in 
the field of ultrasound medical imaging. 

Matching one shape with respect to another is often needed in particular to compare 
medical images together. The amount of calculations needed is often problematic when 
dealing with 2D or 3D images or when a time sequence of such images has to be processed. 

15 The need for computationally efficient and accurate procedures is most acute when dealing 
with real-time applications. 

Background of the Invention 
An Image Registration Technique is already described by T. Netsch, P. Rosch, A. van 
Muiswinkel and J. Weese in the publication entitled "Towards real-time multi-modality 3-D 

20 medical image registration", Computer Vision, 2001. ICCV 2001, Proceedings of Eighth 
IEEE International Conference, vol.1, pp. 718 -725 (2001). This cited Reference compares 
Medical Image Registration based on "Mutual Information" and "Local Correlation. The 
"Local Correlation" based method produces better or superior results to those of "Mutual 
Information", and besides requires considerably fewer computations. The "Local Correlation" 

25 used in the method is a similarity measure of gray- levels within pairs of small regions, 

defined by "windows", in two images to be matched together. The registration then consists 
in looking for the transformation of one image that will result in the largest possible value of 
the sum of such "Local Correlations" over pairs of corresponding windows in the two images. 
Different strategies are proposed in the cited publication to select the windows in each of the 

30 images being compared and to select which pairs of windows have to be compared. The 

choice of the strategy basically depends on the nature of transformation needed, called rigid 
or non-rigid transformations. 

Even though results obtained with the "Local Correlation" based technique are 
promising, computation times are still a problem when dealing with large image data, in 
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particular in the case of image processing in real time and in 3-D. Furthermore, robustness 
has to be improved in particular when dealing with non-rigid registration. To avoid matching 
together local shapes, which are only partly similar, the windows to compare with each other 
have to be large. In this case, each "Local Correlation" evaluation is computation intensive 
5 because it often involves image warping procedures requiring complex numerical algorithms. 

Summary of the Invention 
It is an object of the invention to propose an image processing system to quickly and 
accurately register one shape relative to another shape that are each defined within an image 
window. This system is applicable to match shapes observable in image signal or image data 
10 with any number of dimensions such as: ID signals, 2D, 3D, 2D+time, 3D+time images. The 
present system further allows of determining the image transformation that provides the best 
match between two shapes that are each defined within a window. It is a further object of the 
invention to provide a system that is especially appropriate to match two objects, having 
complex shapes. This system is applicable to set in correspondence one part of the complex 
15 shape of a first image with one part of the complex shape of a second image in order to 
perform local correlation of said parts of the first and second complex shapes. 

The main means of this system are recited in Claim 1 . Specific embodiments of the 
invention are claimed in dependent Claims. An image processing method is proposed to be 
operated in this system. 

20 The medical image processing system can be implemented as a specially programmed 

general-purpose computer. The medical image processing system can be a workstation. The 
present invention yet further provides a computer program product having a set of 
instructions, when in use on a general-purpose computer, to cause the computer to perform 
the steps of the method. The present invention still further provides a medical examination 

25 apparatus incorporating medical image processing system putting into practice the method to 
process medical image data obtained by the imaging apparatus, and means for visualizing the 
image data produced by the method. The visualization means typically consists of a monitor 
connected to the data processing apparatus. Advantageously, the workstation and medical 
imaging system of the present invention are interactive, allowing the user to influence clinical 

30 data that are evaluated and/or the manner in which evaluated data is to be visualized. 

Brief Description of the Drawings. 
The invention and additional features, which may be optionally used to implement the 
invention to advantage, are described hereafter with reference to the schematic figures, 
where: 
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FIG.l is a functional block diagram of the correlation means of the imaging system of 
the invention; 

FIG.2A and FIG.2B illustrate the matching of two shapes using the correlation means; 
FIG.3 is a functional block diagram of a particular embodiment of the correlation 
5 means of the imaging system of the invention; 

FIG.4 is a functional block diagram of a medical examination apparatus using the 
imaging system of the invention. 

Description of Embodiments 
An image processing method is first described. This method has steps of correlating 
10 features, hereafter called shapes, in medical images. The invention further relates to a 

medical image processing system having means for carrying out this method and having 
viewing means to visualize images. The invention also relates to a medical examination 
apparatus having such an image processing system. 

The invention defines an image processing method to find a similarity measure 
15 between two shapes each defined within a Gaussian window. It provides the key components 
for quickly and accurately matching two objects having complex shapes in medical images. 
The processing method requires prior computation of "Hermite Transforms" of the shapes to 
match together within their respective windows. This operation allows of representing each 
of the shapes to be matched by a set of " scalar valued Hermite Transform coefficients 1 '. 
20 These scalar valued Hermite coefficients can be ordered in such a way that low-order 

coefficients code the coarse shape information whereas the high order coefficients code fine 
details. The new proposed processing method provides a way to compute the correlation 
between the shapes to match in a computationally simple manner by combining the scalar 
valued Hermite Transform coefficients related to the two shapes. This allows of 
25 computationally efficiently determining the best matching transformation. The computation 
burden is particularly small if coarse transformation estimates are required. It is particularly 
suited for a coarse to fine registration approach. 

hi the new processing method, the class of transformations, which can be 
simultaneously dealt with, are: 
30 for any data dimension: translation, scale-change; and, in addition 

for data dimensions higher than ID: rotation, anisotropic scale change. 

This method is especially appropriate to match two objects having complex shapes. 
This method is applicable to set in correspondence one part of the complex shape of the first 
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object with one part of the complex shape of the second object in order to perform a local 
correlation of said parts of the first and second complex shapes. 

The processing method is suited to match shapes in medical images, which either 
come from a single imaging modality, modalities, such as Ultrasound, X-ray, MRI or Nuclear 
5 Isotope Imaging, or to match together shapes observable in images of different modalities. 
The present invention preferably makes use of mathematical tools based on a 
polynomial functions such as the Hermite transform. Said Hermite Transform allows the 
analysis of a 1 -dimensional image signal (1-D) or of a m-dimensional image signal (m-D) 
with m>l within an isotropic Gaussian Window characterized by a size parameter a. This 

10 allows to represent the data by a set of " scalar valued Hermite Coefficients" which can be 
ordered in such a way that low order coefficients represent coarse, or low frequency, shape 
information within the Gaussian windows, whereas high order coefficients represent finer 
details, or high frequencies. Essentially, a Hermite Transform allows to reconstruct the 
original signal or image data within a Gaussian window with very small error near the center 

15 of the Gaussian window, for example within a distance less than 2a from the window center. 
The steps of the method of the invention are illustrated by the block-diagram of 
FIG.l. FIG.l is a block diagram for the computation of the windowed correlation function 
K(v) for image signals f(x) and g(y) respectively. Each of the two image signals is defined 
within a given Gaussian window Wl, W2 around center points x c and £ c respectively. The 

20 sizes of the two windows Wl, W2 are defined by the scale parameters ai and a 2 

respectively. HT stands for taking the Hermite Transform to convert a scalar valued signal 
f(x) or g(x) into a set of scalar valued Hermite coefficients. This image processing method 
can apply to image signals depending on one independent variable corresponding to 1- 
dimensional (1-D) image or can apply to image signals having m independent variables 

25 corresponding to m- dimensional images (m-D images). The scalar valued Hermite 

coefficients depend on indices I, which are either simple non-negative integers in 1-D or 
multiple indices (vectors of m-integer indices). The boxes marked rotation Ri or R 2 are 
applicable only for m>l. They convert the set of scalar valued Hermite coefficients into a 
new set corresponding to a rotated version of the corresponding image. The box marked 

30 HT^ 1 takes the scalar valued Hermite correlation coefficients Ki and produces the correlation 
function K(v) computed to a desired accuracy. Best matching occurs when the translation 
parameter v together with those defining relative scaling factors { z k } as well as the rotations 
Ri and R2 of image signals f(x) and g(y) result in the highest value of K(V). The steps of this 
method are described hereafter in detail, comprising: 
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In an image, definition of the image intensity fin function of the location of each 
image pixel or image voxel. So, the scalar value image intensity f is given as function of the 
real positional coordinates. In the present method of the invention, for correlating two shapes 
found in two images or in two parts of an image, two m-dimensional signals or image 
5 intensities f(x) and g(y), are respectively defined in steps referenced 11,21, with m>l. 

Besides, a positive valued function is introduced to define a fuzzy observation 
window W in the image. Each signal or image intensity is defined within a respective 
window called first window Wl and second window W2. The windows Wl, W2 are 
respectively centered around points x c and £ c . The windows are each defined by respective 
10 size parameters ai and ct 2 . A fuzzy Gaussian window function is associated with each of 
these windows, which is maximum near the window center and decays proportionally to 
exp(-(d q /<r q ) 2 ) when moving away from the window center where d q is the distance from the 
respective window center, where subscript q is 1 for the first window and 2 for the second 
window. 

15 The present method has further steps to form a correlation algorithm as follows: 

Steps referenced 12, 22, for evaluation of the Hermite Transform, denoted by HT, 
applied to f(x) and g(x) each in its respective window Wl, W2. This operation results in two 
sets of scalar valued Hermite Transform coefficients { f i } and { gi } respectively, referenced 
13, 23, where each coefficient is labeled with an index I, which is just a non-negative integer 

20 (i) for 1-D signals where m=l, or is a multiple index I=(ii,. . .,i m ) for multidimensional image 
data where m>l. 

Steps referenced 14, 24 for operating rotations. For data dimension m larger than 1, 
the image signals f(x) and g(x) may be rotated around their respective window center x c , Xc 
using respectively (mxm) rotation matrices Ri and R 2 . According to the invention, these 

25 rotations are not operated on the original image signals f(x) and g(x). Instead, they are 

operated on the scalar valued Hermite Transformed Coefficients { f i } and { g i } respectively 
making use of the fact their values convert under rotation like tensors. 

FIG.2 illustrates the principle to find rotation matrices Ri and R 2 to apply to image 
f(x) and g(y) about their respective window center as part of the general processing method 

30 outlined in FIG.l. The case illustrated is for 2-D images (m=2) and when anisotropic scale 
change is required for matching the two shapes. Rotations Ri and R 2 are those which align 
shape principal directions of respective image signals f(x) and g(y) with predefined co- 
ordinate axes. 
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The optimum determination of the rotation matrices Ri and R 2 can be done in 
different ways according to the application, in particular: 

If anisotropic scaling has to be introduced for matching the two shapes 
corresponding to the image signals f(x) to g(x) ? the principal directions of each shape within 
5 its window are determined and the images are rotated so that the principal directions coincide 
with the predefined axes of co-ordinates as schematically shown in FIG.2. 

If no anisotropic scaling needs to be performed, one of the two rotations may 
be set to the identity (no rotation) and the other is determined as part of the general 
optimization processing method described with respect to the following step referenced 31 
10 below. 

Operations referenced 14, 24, which permit of estimating new converted scalar valued 
Hermite coefficients { Fi } and { Gi } referenced 15, 25 that are respectively linear 
combinations of the old coefficients {ft } and { gi }. The linear conversion from the set of 
scalar valued Hermite coefficients {ft } to the set { Fi } is calculated by standard 
15 techniques using the first rotation matrix Ri. Likewise, the conversion from the set of scalar 
valued Hermite coefficients 

{ g i } to { Gi } makes use of the second rotation matrix R 2 . 

In step referenced 31, for the situations where either anisotropic or isotropic scaling is 
needed, performing a computation of the combination of the scalar valued Hermite 
20 coefficients { Fj } and { Gi } using a combination function. This computation provides a 
new set of Hermite Transform Coefficients Ki that will further be used to compute the 
wanted correlation function K(v). The combination formula for computing the Hermite 
Transform Coefficients Ki can favorably be as follows: 



30 



F,G W (1) 



{j |l-J=valid} k=0 ^ UlJ 

25 where the summation extends over all multi-indices defined by: 

J=Gl>j2v.- ? jm) (2a) 

(I-J) = ( ft-ji, i 2 -j2v ? im-jm) (2b) 
where J and (I-J) are valid indices or multi-indices, for which all entries are non-negative 
integers, while 



k 



stands for a binomial coefficient, and: 



(3) 
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where z k is the scaling factor to apply to the co-ordinate number k of 
(x-x c ); and where: 

a k = 1 /Vl + zJ (4a) 

0 k = z k /Vl + z^ (4b) 

Hence, if anisotropic scaling is needed, the coefficients oc k? p k permit of changing 
scale along all the different axes of co-ordinates. 

If isotropic scaling is needed, this is a special case for which all z k values are equal. 

If no scaling is needed, this is another special case for which all z k values are equal to 
1 resulting in that: oc k = p k = l/yfl . 

In step referenced 32, using Hermite Transform Coefficients { Ki }, the inverse 
Hermite Transform (HT" 1 ) allows to estimate a further set of scalar coefficients of a function 
called correlation function K(v). 

In step referenced 33, the correlation function K(v) is computed, where v is a set of 
translation parameters. 

The finally wanted similarity measure and the corresponding optimum translation 
parameters v are given by the maximum value of the correlation function K(v) estimated 
among all calculated values. 

The number of Hermite coefficients Ki to use and the set of translation parameters 
y for which K(y) is computed can be determined according to the desired accuracy. For a 
coarse estimate, only low order indices (I) should be used and only few sampling points for v 
are needed. 

Particular add-on technique: In certain cases, a more accurate determination of the 
relative scale-factors z k , z f k is desired. The correlation measure K(y) may not be sufficiently 
sensitive to scale-change for yielding accurate scale-factors z k , z k estimate. A particular add- 
on technique to be applied to the above method and related processing system is proposed. 

Given a "warping law" defined by some combination of anisotropic scale change 
translation and rotation, the add-on technique consists in deducing the Hermite Transfonn of 
the warped pattern from the Hermite Transform of the pattern before warping. For comparing 
patterns, the proposed add-on technique consists in deducing a "difference measure" of 
patterns from the difference of their Hermite Transform coefficients. In this way, the warping 
parameters that result in the best match between the compared patterns can be determined as 
follows, hi this add-on technique, all "warping parameters", including rotation and 
translation, are determined using the previously described procedure for determining the 
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coefficients Fi, Gi. From the coefficients F h Gi, then a quadratic difference measure is 
defined based on said Hermite coefficients Fi, Gi. The choice of the scale-factors z k , z f k is the 
one that minimizes this quadratic difference measure, hereafter denoted by Pattern Disparity 
Measure D given by the formula: 



where F'i and G'i are derived from the Hermite coefficients Fi and Gi according to the 
technique illustrated by FIG. 3. 

FIG. 3 is a block diagram illustrating the add-on procedure needed when a reliable 
estimate of the best-match relative scales ({ z k },{ z' k } need to be evaluated. The add-on 
procedure requires only the Hermite coefficients F i and Gi determined in steps 15, 25. 

The effect of the relative scaling factors { z k }and { z ? k } and, if needed, of translation 
changes, ( Ax k },{ Ax' k } is to modify the Hermite coefficients through linear operations to 
yield F'i and G'i . The linear operations can be matrix operations. 

The unknown parameter values are computed as the combination of values, among 
the possible combinations of { z k }and {z' k }, ( Ax k }, { Ax' k }, which minimizes the Pattern 
Disparity Measure D, given by (6), since D is a function of: 



The above combination of steps, from 12, 22 to 33 can be repeated several times if 
necessary to reach the best possible similarity measure by modifying the rotation matrices 
Ri , R 2 , the scaling factors { z k } and by increasing the number of indices I, J if more 
accuracy is needed. A standard strategy may be used for this optimization. 

Once this optimization is done, one may derive the best transformation that will 
provide the best match of one image to the other under the matrix equation form: 



D = Z C, (F'j-G'j) 2 



(6) 



i 



({z k },{z' k },({Ax k },{Ax' k }). 




(5a) 



and: 




where Z x and Z 2 are both (mx m) diagonal matrices with: 



matrix Zi has diagonal element at row and column k, equal to a 2 z k /ai, and 
matrix Z 2 has diagonal element at row and column k 9 equal to oi/^Zfc); 



and where A and B are also both (mx m) diagonal matrices with: 



matrix A has diagonal element at row and column k, equal to a 2 /oc k? and 
matrix B has diagonal element at row and column k equal to <Ti/p k? 
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where a 1? a 2 , z k , a k , p k , v are defined above. 

Even more economy in computation load can be achieved when one is interested in 
studying a restricted class of image transformations. 

In particular, if only translations in the direction defined by co-ordinate number n is 
5 involved, the above evaluations of {Ki}, can be limited only to multi-indices I=(ii v ..,i m ) for 
which i k =0 for all co-ordinate numbers k^n. This allows to compute a one dimensional 
correlation function K(v n ) depending on the one dimensional co-ordinate v n of the m- 
dimensional translation parameter v .Considerably less computations are then needed to find 
the optimum of K(v„) as compared to the calculations needed to find the optimum of K(v) 
10 which has an m-dimensional independent variable. 

The present invention further relates to a viewing system having processing means for 
carrying out the steps of the image processing method and technique of the invention and 
having display means to visualize the processed images. 

Hence, the image processing system of the invention has means for computing the 
15 correlation or degree of similarity of two image signals, each signal being defined within a 
respective isotropic Gaussian window. More specifically, the present system relates to 
processing means for: 

estimating in 1 1, 21, two image signals f(x), g(y) representing shapes defined in 
respective windows Wl, W2 in two multi-dimensional images (m-D); 
20 using a Hermite Transform HT in 12, 22, applied to both said image signals f(x), g(x) 

for perfonning an evaluation of two first sets of scalar valued Hermite coefficients ft, g l9 F b 
Gi in 13, 15, 23, 25, from which a combination 3 1 yields a transformed set of scalar valued 
Hermite coefficients {Ki}, with scaling factors {z k }; 

applying the inverse Hermite Transform HT" 1 in 32 to said transformed set of scalar 
25 valued Hermite coefficients {Ki} to achieve the computation of said windowed correlation 
function K(v) in 33; 

means to estimate the wanted similarity measure as the maximum of the correlation 
function K(v); and 

means for displaying the correlated shapes and/or processed images. 
30 Compared with the state of the art techniques, this current method and system deal 

with the following issues: 

In addition to translation, which is handled by means 31 of computing the scalar 
valued Correlation Coefficients {Ki}, and/or in addition to rotation, which is handled by 
rotation matrices Ri and R 2 , respectively in 14 and 24, the system of the invention has means 
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33 of computing the correlation function K(v) ? which handles change of scale. This means 33 
allows of estimating the effect of altering scale (isotropically or not) in one image signal 
relative to the other, while, at the same time, taking into account the effect of translation and 
rotation. This is a unique feature, which is not present in other state of the art approaches to 
5 compute local correlation. 

Accurate registration is possible with fewer computations than for state of the art 
methods in particular when large correlation blocks need to be used. 

The amount of calculations required to obtain the correlation function is adaptable to 
the accuracy of match that is required. The number of Hermite coefficients Ki to use and the 
10 set of translation parameters v for which K(v) is computed are determined according to the 
desired accuracy. This method and system permit of calculating the correlation function K(v) 
to a desired accuracy in sub-sampling the set of possible translation parameters v for which 
the correlation is to be computed. For a coarse estimate, only low order indices (I) and only 
few sampling points for v are used. This leads to very few calculations if a coarse match is 
15 needed, which also minimizes calculations for a "coarse to fine approach". 

When the correlation function K(v) is evaluated in multi-dimensional images, it may 
be calculated for a translation parameter v varying in a direction parallel to one coordinate 
axis at a time with great saving in computations. 

The method and system present a possibility to remove the effect of the local mean 
20 intensities from the correlation for example by skipping multi-indices which have all zero 
entries in the Hermite Transforms of both images. 

An important issue that is dealt with by the method and system is a possibility of 
matching images with inverse contrast. This problem occurs when images coming from 
different sources such as x-rays and MRI are considered. This problem is solved by looking 
25 for both positive maxima AND negative minima of K(v) and using the absolute value 

| K(v) | instead of K(v). This aspect is important when matching such images coming from 
different modalities in which case the contrast of an anatomical feature in one image may be 
inverted with respect to that of the other. 

These method and system present a possibility of matching features which have 
30 different amplitudes in the two images by dividing K(v) or | K(v) | by the product of local • 
root mean square image amplitudes. 

Better robustness can be obtained against noise as compared to state of the art. The 
techniques of the state of the art need to warp one of the images relative to the other even for 
local correlation. This may produce artifacts and is computationally costly. This is totally 
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avoided in the method and system of the invention for local correlation. Such warping 
operations are replaced by much simpler manipulations of the Hermite coefficients of the 
images and of the correlation function. 

The above-described image viewing method, which is based on a windowed 
5 correlation technique, can be used as a key component of a freeform or elastic matching 
technique. One possible version of such technique is described hereafter as an example. As 
illustrated by FIG.2B, a complex shape called SHAPE 1 must be matched to a complex shape 
called SHAPE 2 represented respectively in a first image and in a second image. For 
matching SHAPE 1 and SHAPE 2, it may be necessary to warp linearly or non-linearly one 
10 of the shapes. The disks shown in the two images of the shapes indicate local windows in 
each image, in which gray level intensities vary appreciably. Each of the images to match is 
paved with a number (preferably large) of such windows. To find the warping law, one can 
proceed in successive iterations as follows: 

a) An initial guess of the warping law is provided (for example using a global 
1 5 alignment procedure known from the state of the art), 

b) For each of the windows Wl, such as Wll, W12, W13, in one of the 
images, called first image, which window is centered around a point x c , the current guess of 
the warping law is used in order to find the corresponding location t in the other image, 
called second image, as well as an approximate anisotropic scale change and rotation within 

20 the window Wl introduced by the warping law using state of the art technique. The locations 

of all windows W2 k are looked for in the second image, having their centers y^ k) within a 

maximum distance from point t, which distance is determined by the algorithm user. If no 
such window exists in the second image, window Wl is discarded. 

c) If the window Wl is not discarded: for each of the windows W2 k in the 
25 second image, the method proposed in the invention is used together with the information 

obtained in above step b) to obtain the correlation between window W2 k in the second image 
and window Wl in the first image. The maximum correlation C k is determined together with 
the corresponding translation At k needed to match W2 k in order to the warped window Wl 
which mapped around point t. 
30 d) The maximum correlation C k is stored as C together with the 

corresponding translation At k as At. 

e) The above two steps (c) and (d) are repeated for all non-discarded windows 
Wl in the first image. From all the translations Dt, modify the warping law to better describe 
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matching of the two images using state of the art techniques. The goodness of fit rate G is 
calculated from the sum of local correlations C over all windows Wl divided by the number 
of non-discarded windows Wl. 

f) Steps b) to e) above are repeated, up to a maximum number of user defined 
5 iterations or when the goodness of fit rate G is considered satisfactory (Le. above a level 

determined by the algorithm user). 

g) The best warping obtained is considered as the one that maximizes the sum 
of local correlations. 

FIG.2B illustrates the scheme for registration using a given deformation law. Circles 

10 symbolize the windows Wl in the first image and the corresponding windows W2 in the 
second image. The arrows symbolize the correspondence between windows that maximizes 
the local correlation. For instance, windows W21, W22, W23 in the second image 
respectively correspond to windows Wll, W12, W13 in the first image. 

The above-described steps can be carried out by the viewing system of the invention. 

15 FIG.4 shows a diagram of a medical viewing system 150 according to the invention for 
carrying out all the steps of the above-described method. The system has means 151 for 
acquiring digital image data of a sequence of images, and is coupled to computer means 153 
for processing these data according to the processing method cited above. The medical 
viewing system can be used in the intervention room or near the intervention room for 

20 processing real time images. Steps of the present method can be applied on stored medical 
images, for example for estimating medical parameters. The medical viewing system 
provides the image data by connection 157 to the system 153. The system provides processed 
image data to display means and/or storage means. The display means 154 may be a screen. 
The storage means may be a memory of the system 153. Said storage means may be 

25 alternately external storage means. 

The data processing means 153 is typically associated with a visualization device, 
such as a monitor 154, and an input device 155, such as a keyboard, pointing device, such as 
a mouse, etc. operative by the user so that he can interact with the system. Icones may be 
provided on the screen to be activated by mouse-clicks, or special pushbuttons may be 

30 provided on the system, to constitute control means 158 for the user to control the processing 
means of the system. The data processing device 153 is programmed to implement the 
method of processing medical image data according to invention. In particular, the data 
processing device 153 has computing means and memory means to perform the steps of the 
method. A computer program product having pre-programmed instructions to carry out the 
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method may also be implemented. This image viewing system 153 may comprise a suitably 
programmed computer, or a special purpose processor having circuit means such as LUTs, 
Memories, Filters, Logic Operators, that are arranged to perform the functions of the method 
steps according to the invention. 
5 The basic components of an image viewing system in accordance to the present 

invention can be incorporated in a medical examination apparatus. As indicated 
schematically in Fig.4, the medical examination apparatus typically includes a bed 1 10 on 
which the patient lies or another element for localizing the patient relative to the imaging 
apparatus. The medical imaging apparatus maybe a CT scanner 151. The image data 
10 produced by the CT scanner 151 is fed to data processing means 153, such as a general- 
purpose computer, that carries out the steps of the method. Or the medical imaging apparatus 
may be an Ultrasound apparatus whose images can be processed using the image processing 
system. 

The present invention is applicable regardless of the medical imaging technology that 
15 is used to generate the initial data. For example, when seeking to visualize the heart, 

magnetic resonance (MR) coronary angiography may be used to generate 3D medical image 
data in a non-invasive manner. Various modifications can be made to the order in which 
processing steps are performed in the above-described specific embodiment. The drawings 
and their description herein before illustrate rather than limit the invention. It will be evident 
20 that there are numerous alternatives that fall within the scope of the appended claims. 

Moreover, although the present invention has been described in terms of generating image 
data for display, the present invention is intended to cover substantially any form of 
visualization of the image data including, but not limited to, display on a display device, and 
printing. Any reference sign in a claim should not be construed as limiting the claim. 
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CLAIMS: 

1 . An image processing system, for correlating shapes in multi-dimensional images (m- 
D), comprising image data processing means for estimating a similarity measure including 
computing means for: 

estimating two image signals (f(x), g(^)) representing shapes defined in respective 
windows (Wl, W2) in two multi-dimensional images; 

using a Hermite Transform (HT) applied to both said image signals for performing an 
evaluation of two first sets of scalar valued Hermite coefficients (ft, gi, Fi, Gi), from which a 
combination yields a transformed set of scalar valued Hermite coefficients {Ki}; 

applying the inverse Hermite Transform (HT -1 ) to the transformed set of scalar valued 
Hermite coefficients {Ki} to achieve the computation of a windowed correlation function 
(K(v)); and estimating the maximum of said windowed correlation function as the wanted 
similarity measure to correlate the shapes; and 

means for displaying the correlated shapes and/or processed images. 

2. The system of Claim 1, comprising data processing means for ordering the scalar 
valued Hermite coefficients (K r , ft, g r ) in such a way that low-order coefficients code the 
coarse shape information whereas the high order coefficients code fine details. 

3. The system of one of Claims 1 or 2, wherein the class of simultaneous 
transformations, which is dealt with for any data dimension m, comprises: 

translation, scale-change. 

4. The system of one of Claims 1 to 3, wherein the class of simultaneous 
transformations, which is dealt with for data dimensions higher than 1-D, comprises: 

translation, scale-change, and, in addition 
rotation and anisotropic scale change. 

5. The system of Claim 4, comprising, for a number of variables superior to one 
corresponding to data dimensions higher than 1-D, data processing means for: 

processing the first two sets of scalar valued Hermite coefficients (ft, gi) by applying 
at least one rotation matrix (Rl 3 R2) which converts at least one of said two sets of scalar 
valued Hermite coefficients (ft, g x ) into a new set of scalar valued Hermite coefficients (Fi, 
d) corresponding to a rotated version of at least one of the shapes; 
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estimating a transformed set of scalar valued Hermite coefficients {IQ} from the new 
sets of scalar valued Hermite coefficients (F& G r ) corresponding to the rotated versions of the 
shapes. 

6. The system of one of Claims 1 to 5, comprising data processing means for: 
setting the number of scalar valued Hermite correlation coefficients {Ki} to use and 

the set of translation parameters (v) for which the correlation function (K(y)) is to be 
computed in order to provide a desired accuracy, according to which for a coarse estimate of 
only low order indices (I) and a limited number of sampling points for translation parameters 
(v) are used, while the maximum value of the correlation function (K(v)), among all 
calculated values, provides the correlation measure and the corresponding optimum 
translation parameter (v). 

7. The system of Claim 6, comprising data processing means for: 

setting the number of scalar valued Hermite correlation coefficients {Ki} to use, when 
only translations in the direction defined by a given co-ordinate number (n) is involved, 
comprising limiting said evaluations of scalar valued Hermite correlation coefficients {Ki} to 
multi-indices I=(*i»—4m) for which i k =0 for all co-ordinate numbers k^n, wherefrom 
computing a one dimensional correlation function (K(v n )) depending on the one dimensional 
co-ordinate (v„) of the m-dimensional translation parameters (v). 

8. The system of Claim 4, comprising computing means for estimating scaling factors 
from the estimation of the Hermite coefficients (Fi, Gi) by defining a quadratic difference 
measure (D) based on said Hermite coefficients (F r , d) and choosing the scale-factors (z k , 
z f k ) as the combination that minimizes this quadratic difference measure. 

9. The system of one of Claims 1 to 8, comprising data processing means for: 
repeating the steps of determining the correlation function, as many times as 

necessary in order to reach the best possible correlation function by modifying the rotation 
matrices (Ri, R 2 ), the scaling factors ({ z k }) and by increasing the number of indices (I) if 
more accuracy is needed. 

10. The system of one of claims 1 to 9, comprising data processing means for performing 
an evaluation of a warping law for correlating a first and a second complex shape in multi- 
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dimensional images (m-D), including the determination of more than one window (Wl) for 
the first shape and the determination of corresponding candidate windows (W2 k ) for the 
second shape, and further comprising data processing means for performing the estimation of 
the Hermite Transform (Ki) for the evaluation of a windowed correlation function K(v) ? 
5 including steps of using a Hermite Transfomi (HT) of two image signals (f(x), g&)) defined 
in respective windows (Wl, W2), wherefrom data processing means for performing an 
evaluation of the inverse Hermite Transform (HT** 1 ) to achieve the computation of said 
windowed correlation function (K (v)), which is used to determine the best candidate 
windows, and comprising data processing means for performing the determination of the best 
1 0 warping law by iteration of these steps in order to match the first and second complex shapes. 

11. An image processing method comprising an evaluation of a Hermite Transform (Ki) 
for the evaluation of a windowed correlation function K(v), including steps of using the 
Hermite Transform (HT) of two image signals (f(x), g&)) representing shapes defined in 
15 respective windows (Wl, W2) in multi-dimensional images (m-D), wherefrom a step of 

evaluating the inverse Hermite Transform (HT -1 ) is performed to achieve the computation of 
said windowed correlation function (K(v)).and 

visualizing the correlated shapes and/or processed images. 

20 12. A medical examination apparatus comprising acquisition means for acquiring medical 
image data, and an image processing system according to one of Claims 1 to 10. 



25 



13. A computer program product having a set of instructions, when in use on a general- 
purpose computer, to cause the computer to perform the steps of the method according to of 
Claim 11. 
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